Asynchronous Trajectory Matching-Based Multimodal Maritime Data Fusion for Vessel Traffic Surveillance in Inland Waterways

The automatic identification system (AIS) and video cameras have been widely exploited for vessel traffic surveillance in inland waterways. AIS data could provide identity dynamic information on position movements. In contrast, the describe visual appearances of moving vessels without knowing identity, position, movements, etc. To further improve surveillance, it becomes necessary to fuse simultaneously capture features, interest. However, performance fusion is susceptible issues such as spatial difference, message asynchronous transmission, object occlusion, this work, we propose a deep learning-based simple online real-time method (termed DeepSORVF). We first extract AIS-and video-based trajectories, then an trajectory matching AIS-based with corresponding targets. addition, by combining movement also present prior knowledge-driven anti-occlusion yield accurate robust tracking results under occlusion conditions. validate efficacy our DeepSORVF, constructed new benchmark dataset FVessel) detection, tracking, fusion. It consists many videos collected various weather conditions locations. experimental demonstrated that capable guaranteeing high-reliable tracking. DeepSORVF code FVessel are publicly available at https://github.com/gy65896/DeepSORVF https://github.com/gy65896/FVessel, respectively.